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Abstract. The referential properties of noun phrases in the Japanese 
language, which has no articles, are useful for article generation in Japanese- 
English machine translation and for anaphora resolution in Japanese 
noun phrases. They are generally classified as generic noun phrases, def- 
inite noun phrases, and indefinite noun phrases. In the previous work, 
referential properties were estimated by developing rules that used clue 
words. If two or more rules were in confiict with each other, the cate- 
gory having the maximum total score given by the rules was selected 
as the desired category. The score given by each rule was established 
by hand, so the manpower cost was high. In this work, we automatically 
adjusted these scores by using a machine-learning method and succeeded 
in reducing the amount of manpower needed to adjust these scores. 



1 Introduction 

To estimate the referential property of a noun phrase (NP) in the Japanese 
language, which does not have any articles, is one of the most difficult problems 
in natural language processing The referential property of a noun phrase 
represents how the noun phrase denotes the referent and is classified into the 
following three types: 

• An indefinite NP — denotes an arbitrary member of the class of the noun 
phrase. 

(Ex.) There are three dogs. 

• A definite NP — denotes a contextually non-ambiguous member of the class 
of the noun phrase. 

(Ex.) The dog went away. 

• A generic NP — denotes all members of the class of the noun phrase or the 
class itself of the noun phrase. 

(Ex.) Dogs are useful. 

Note that "dogs" in this sentence denotes general dogs and are classified 
generic. 



Estimating the referential properties of noun phrases in Japanese sentences 
is useful for (i) generating articles when translating Japanese nouns into English 
and (ii) estimating the referents of noun phrases. 



(i) Article generation in machine translation 

In the process of generating articles, when a noun phrase is estimated to 
be indefinite, it is given the indefinite article, "a/an" , when it is singular, 
but is given no article when it is plural. When a noun phrase is estimated 
to be definite, it is given the definite article, "the" . When a noun phrase is 
estimated to be generic, usage in terms of articles is generated by a method 
used for generic noun phrases (a generic noun phrase can be given a definite 
or an indefinite article or no article, and may also be in the plural form.)|^ 
For example, hon (book) in the following sentence is a generic noun phrase. 

hon-toiunowa ningen-no seichou-ni kakasemasen 

(book) (human being) (growth) (be necessary) (1) 
(Books are necessary for the growth of a human being.) 

So it can be translated as "a book," "books," or "the book" in English. Note 
that in the following sentence, hon (book) is a definite noun phrase. 

kinou boku-ga kashita hon-wa yomimashitaka 
(yesterday) (I) (lend) (book) (read) (2) 
(Did you read the book I lent you yesterday?) 

It can thus be translated as "the book" in English. 

(ii) Anaphora resolution 

Only a definite noun phrase can refer to a previous noun phrase and this is 
very useful in anaphora resolution j^] . For example, in the following example, 
hon (book) in the second sentence is a generic noun phrase, so it cannot be 
referring to hon (book) in the first sentence. 



hon-wo omiyage-ni kaimashita. 
(book) (as a present) (buy) 
(I bought books as a present.) 

(3) 

hon-toiunowa ningen-no seichou-ni kakasemasen 
(book) (human being) (growth) (be necessary) 
(Books are necessary for the growth of a human being.) 

As in the above explanation, the referential properties of noun phrases, i.e., 
generic, definite, and indefinite, are useful for article generation and anaphora 
resolution, and estimating them is a serious problem in natural language pro- 
cessing. 

^ Bond et al. have actually used the referential properties of noun phrases in gener- 
ating articles [^. 



In the conventional estimation of referential properties g, heuristic rules 
(made by hand) using surface clue words are used for estimation. For example, 
in sentence above the referential property is estimated to be generic by using 
a Japanese clue word toiu-nowa; in sentence (^), the referential property of a 
noun phrase, hon (book), is estimated to be definite since the noun phrase is 
modified by an embedded sentence, kinou boku ga kashita (I lent you yesterday). 
In their work, 86 heuristic rules were created. When plural rules conflicted and 
the rule used in estimation was ambiguous, the conflict was solved by using the 
scores given in the rules. These scores needed to be adjusted by hand in order 
to properly resolve conflicts. 

In the current work, to reduce human costs of previous research, we have used 
a machine-learning method to automatically adjust these rules. We selected the 
maximum entropy method (which is robust for sparse data problems) as the 
machine-learning method. 



2 How to estimate referential property 
2.1 Method used in previous research 

The previous research gave each referential property two evaluation values, pos- 
sibility and value, by using heuristic rules and estimated the referential property 
according to these values. Here, possibility is logically conjuncted and value is 
added. As a result, the referential property whose possibility is 1 and whose value 
is maximum is estimated to be the desired one. 
Heuristic rules are given in the following forms: 

{condition for rule application) 

=4> { indefinite {possibility, value) 
definite {possibility, value) 
generic {possibility, value) } 

A surface expression, which contains a clue word for estimating the referential 
property, is written in condition for rule application. Possibility has a value of 
1 when the categories indefinite, definite and generic are possible in the context 
checked by the condition. Otherwise, the possibility value is 0. Value means that 
a relative possibility value between 1 and 10 (an integer) is given according to 
the plausibility of the condition that the possibility is 1. A larger value means 
the plausibility is high. 

Several rules can be applicable to a specific noun in a sentence. In this case, 
the possibility values for the individual categories are added, and the category 
for the noun is decided as the category with the highest sum of possibility values. 

86 rules were created. Some of the rules are given below. 



(1) When a noun is modified by a referential pronoun, kono (tliis), sono (its), 
etc., tfien {indefinite (0, 0)0 definite (1, 2) generic (0, 0)} 

kono hon-wa omoshiroi. 
(this) (book) (interesting) 
f This book is interesting.) 

(2) When a noun is accompanied by a particle, wa, and the predicate is in the 
past tense, then {indefinite (1, 0) definite (1, 3) generic (1, 1)} 

inu-wa mukou-he itta. 
(dog) (away there) (went) 
(The dog went away.) 

(3) When a noun is accompanied by a particle, wa, and the predicate is in the 
present tense, then {indefinite (1, 0) definite (1, 2) generic (1, 3)} 
inu-wa yakunitatsu doubutsu desu. 

(dog) (useful) (animal) (is) 
(Dogs are useful animals.) 

When there are no clues, "indefinite" is assigned as the default value. 
Let us look at an example of a noun to which several rules apply, kudamono 
(fruit) as used in the following sentence. 

wareware-ga kinou tsumitotta kudamono-wa aji-ga iidesu. 

(we) (yesterday) (picked) (fruit) (taste) (be good) (4) 
( The fruit that we picked yesterday tastes delicious.) 

All the rules were applied and the condition only satisfied the following seven 
rules which were then used to determine the degree of the definiteness of the 
noun. 

(a) When a noun is accompanied by wa, and the corresponding predicate is not 
in the past tense 

(kudamono-wa aji-ga iidesu ), then 
{indefinite (1, 0) definite (1, 2) generic (1, 3)} 

(b) When a noun is modified by an embedded sentence which is in the past tense 
(tsumitotta) , 

then 

{indefinite (1, 0) definite (1, 1) generic (1, 0)} 

(c) When a noun is modified by an embedded sentence which has a definite noun 
accompanied by wa or ga (wareware-ga), then 

{indefinite (1,0) definite (1, 1) generic (1, 0)} 

(d) When a noun is modified by an embedded sentence which has a definite noun 
accompanied by a particle (wareware-ga), then 

{indefinite (1,0) definite (1, 1) generic (1, 0)} 

(e) When a noun is modified by a phrase which has a pronoun (wareware-ga), 
then 

{indefinite (1, 0) definite (1, 1) generic (1, 0)} 
^ (a, b) means possibility (a) and value (b). 



(f) When a noun has an adjective as its predicate (kudamono-wa azi-qa iidesu ), 
then 

{indefinite (1, 0) definite (1, 3) generic (1, 4)} 

(g) When a noun is a common noun (kudamono), 
then 

{indefinite (1, 1) definite (1, 0) generic (1, 0)} 

By using the scores from these rules, total possibilities and values were cal- 
culated. The overall possibilities for all three categories were 1 because all three 
categories carried possibilities of 1 for all of these rules. The total values for 
all three categories were 1, 9, and 7, respectively, because the results of aggre- 
gating the values of all of these rules were 1 (=0-|-0-|-0-|-0-|-0-|-0-|-l), 9 
(= 2-hl-hl-hl-hl-h3-f 0), and 7 (= 3-h0-H0-h0-H0-h4-h0),for the respective 
categories. A final score of {indefinite (1, 1) definite (1, 9) generic (1, 7)} was 
obtained, and the system judged, correctly, that the noun here is "definite." 

Each noun, from left to right, in the sentence was estimated according to (a) 
- (g) above. This process allows the decision process to make use of referential 
properties that has already been determined (see (c) and (d), for example). 

In the method used in the previous work possibility and value had to be 
adjusted in order to estimate referential properties properly, and this required 
much work by human. Although gathering the clue words by hand might be 
effective, possibility and value can be adjusted by a certain machine-learning 
method. In this paper, we report on our use of the machine-learning method 
described in the next section, to verify this possibility. 



2.2 The machine-learning method 

A machine-learning method was applied to the estimation of referential prop- 
erties. We used the maximum entropy method as a machine-learning method, 
which is robust against data sparseness, because it is difficult to make a large 
corpus tagged with referential properties. By defining a set of features in ad- 
vance, the maximum entropy method estimates the conditional probability of 
each category in a certain situation of the features, and it is called the maximum 
entropy method since it maximizes entropy when estimating a probability. The 
process of maximizing entropy can make the probabilistic model uniform, and 
this effect is the reason that the maximum entropy method is robust against data 
sparseness. We used Ristad's system [^,|| as the maximum entropy method. The 
three probabilities, generic, definite, or indefinite, are calculated from the output 
of Ristad's system. The category having the maximum probability is judged to 
be the desired one. 

To use the maximum entropy method, we must choose the features used 
in learning. We used the conditions of the 86 rules that had been used in the 
previous work. 86 features are thus used in learning. 



If we use, for example, rules 1, 2, and 3 as described in Sec. 2.1, only condition 



parts are detected and the following three features are obtained. 



1. Whether or not a noun is modified by a referential pronoun, kono (this), 
sono (its), etc. 

2. Whether or not a noun is accompanied by a particle wa, and the predicate 
is in the past tense. 

3. Whether or not a noun is accompanied by a particle wa, and the predicate 
is in the present tense. 

Now, we use the last example from the previous section to explain how the 
referential property is estimated by the maximum entropy method. 

wareware-ga kinou tsumitotta kudamono-wa aji-ga iidesu. 

(we) (yesterday) (picked) (fruit) (taste) (be good) (5) 
( The fruit that we picked yesterday tastes delicious.) 

We again look at kudamono (fruit). The same seven rules are again applied. 
The value assigned to each of the referential properties for each rule indicates 
the conditional probability of that category being correct when only that rule is 
applied and they are calculated by the maximum entropy method. The values 
written here were obtained in our experiment of "Machine-Learning 2" described 
in Sec. |. 

(a) When a noun is accompanied by wa and the corresponding predicate is not 
in the past tense, 

( kudamono- wa aji-ga iidesu ), then 
{indefinite 0.31 definite 0.29 generic 0.40} 

(b) When a noun is modified by an embedded sentence which is in the past tense 
{tsumitotta), 

then 

{indefinite 0.31 definite 0.49 generic 0.19} 

(c) When a noun is modified by an embedded sentence which has a definite noun 
accompanied by wa or ga {wareware-ga), then 

{indefinite 0.19 definite 0.61 generic 0.19} 

(d) When a noun is modified by an embedded sentence which has a definite noun 
accompanied by a particle {wareware-ga), then 

{indefinite 0.01 definite 0.80 generic 0.18} 

(e) When a noun is modified by a phrase which has a pronoun {wareware-ga), 
then 

{indefinite 0.20 definite 0.44 generic 0.37} 

(f) When a noun has an adjective as its predicate {kudamono-wa azi-g a iidesu), 
then 

{indefinite 0.13 definite 0.80 generic 0.07} 

(g) When a noun is a common noun {kudamono), 
then 

{indefinite 0.72 definite 0.15 generic 0.14} 

In the maximum entropy method the values assigned by the above rules are 
multiplied, the values in each category are normalized, and the category with 



the highest value is judged to be the desired one. In this case, we multipHed and 
normaUzed the values of all the rules and obtained the following results: 
{indefinite 0.001, definite 0.996, generic 0.002} 

"Definite" had the highest value and was thus judged to be the desired category. 



3 Experiment and Discussion 

Morphological and syntactic information are used as features in estimating ref- 
erential properties. Before estimating the referential property, morphology and 
syntax were analyzed |^,^. We used the same learning set and test set as had 
been used in previous work.^ The 86 rules had been made by examining the 
learning set by hand. In the previous work, the values written in the 86 rules 
had been adjusted by checking the accuracy rates in the learning set. 
Firstly, we carried out the following two experiments. 

— Manual Adjustment — The estimation was made by using the method de- 
scribed in Sec (This result is identical to the result of the previous work.) 

— Machine-Learning 1 — The estimation was made by using the method de- 



scribed in Sec 



2.2 



The results are listed in Tables |i| to ^. "Other" in the tables indicates that 
the referential property is ambiguous; such cases can be neglected here since 
they were few. 

The accuracy rate obtained by Manual Adjustment (Table ||) in the test set 
was 68.9% and that of Machine-Learning 1 (Table ^) was 72.1%. The machine- 
learning method was thus more accurate than Manual Adjustment method. But, 
as can be seen in Tables || and ^, all the categories of Manual Adjustment are 
about 70% and the method did not have a bad-accuracy category. However, the 
result from Machine-Learning 1 of the "generic" category was low (5.2%). We 
cannot therefore conclude that Machine-Learning 1 reliably gives good result^ 

We felt that the reason for this low accuracy in estimating the "generic" 
category is that the frequency of "generic" terms is low and machine learning 
is biased toward "definite" terms, which have higher frequency than "generic" 
terms. We therefore carried out the following further experiments. 

— Machine-Learning 2 — When machine learning is performed by using the 
maximum entropy method, the number of events in each category of the 



The learning set: "Usage of the English Articles" [g] , a folktale "The Old Man with 
a Wen" and an essay tensei jingo. The test set: a folktale turu no ongaeshi 
0, an essay tensei jingo, "Pacific Asia in the Post-Cold- War World" (A Quarterly 
Publication of The International House of Japan Vol. 12, No. 2 Spring 1992). 
Here, we assume that not producing a bad-accuracy category was more important 
than having the highest total accuracy rate. We then constructed Method 2 as 
described in the following passages. However, if we assume that having the highest 
total accuracy rate is more important than not producing a bad-accuracy category. 
Method 1 is, in fact, better. 



Table 1. Manual Adjustment (learning set) 





indef 


def 


gen 


other 


total 


Usage of the Articles 


140 sentences, 380 nouns) 


correct 


96 


184 


58 


1 


339 


incorrect 


4 


28 


8 


1 


41 


% of correct 


96.0 


86.8 


87.9 


50.0 


89.2 


The Old Man with 


a Wen (104 sentences, 267 nouns) 


correct 


73 


140 


6 


1 


222 


incorrect 


14 


27 


4 





45 


% of correct 


83.9 


84.0 


60.0 


100.0 


83.2 


an essay tensei jingo (23 sentences, 98 nouns) 


correct 


25 


35 


16 





76 


incorrect 


5 


14 


3 





22 


% of correct 


83.3 


71.4 


84.2 




77.6 


average 












% of appearance 


29.1 


57.7 


12.8 


0.4 


100.0 


% of correct 


89.4 


84.0 


84.2 


66.7 


85.5 



Table 2. Manual Adjustment (test set) 





indef 


def 


gen 


other 


total 


a folktale Turu (263 sentences, 699 nouns) 


correct 


109 


363 


13 


10 


495 


incorrect 


38 


160 


6 





204 


% of correct 


74.2 


69.4 


68.4 


100.0 


70.8 


an essay tensei jingo (75 sentences, 283 nouns) 


correct 


75 


81 


16 





172 


incorrect 


41 


60 


10 





111 


% of correct 


64.7 


57.5 


61.5 




60.8 


Pacific Asia (22 sentences, 192 nouns) 


correct 


21 


108 


11 


2 


142 


incorrect 


17 


31 


2 





50 


% of correct 


55.3 


77.7 


84.6 


100.0 


74.0 


average 












% of appearance 


25.6 


68.4 


4.9 


1.0 


100.0 


% of correct 


68.1 


68.7 


69.0 


100.0 


68.9 



Table 3. Machine Learning 1 (learning set) 





indef 


def 


gen 


other 


total 


Usage of the Articles 


140 sentences, 380 nouns) 


correct 


95 


199 


32 





326 


incorrect 


5 


13 


34 


2 


54 


% of correct 


95.0 


93.9 


48.5 


0.0 


85.8 


The Old Man with 


a Wen (104 sentences, 267 nouns) 


correct 


71 


151 


1 





223 


incorrect 


16 


18 


9 


1 


44 


% of correct 


81.6 


89.4 


10.0 


0.0 


83.5 


an essay tensei jingo (23 sentences, 98 nouns) 


correct 


21 


46 


5 





72 


incorrect 


9 


3 


14 





26 


% of correct 


70.0 


93.9 


26.3 




73.5 


average 












% of appearance 


29.1 


57.7 


12.8 


0.4 


100.0 


% of correct 


86.2 


92.1 


40.0 


0.0 


83.4 



Table 4. Machine Learning 1 (test set) 





indef 


def 


gen 


other 


total 


a folktale Turu (263 sentences, 699 nouns) 


correct 


104 


408 








512 


incorrect 


43 


115 


19 


10 


187 


% of correct 


70.8 


78.0 


0.0 


0.0 


73.3 


an essay tensei jingo (75 sentences, 283 nouns) 


correct 


72 


108 


2 





182 


incorrect 


44 


33 


24 





101 


% of correct 


62.1 


76.6 


7.7 




64.3 


Pacihc Asia (22 sentences, 192 nouns) 


correct 


21 


130 


1 





152 


incorrect 


17 


9 


12 


2 


40 


% of correct 


55.3 


93.5 


7.7 


0.0 


79.2 


average 












% of appearance 


25.6 


68.4 


4.9 


1.0 


100.0 


% of correct 


65.5 


80.5 


5.2 


0.00 


72.1 



Table 5. Machine Learning 2 (learning set) 





indof 


dcf 


gen 


other 


total 


Usage of the Articles (140 sentences, 380 nouns) 


correct 


97 


188 


57 





342 


incorrect 


3 


24 


9 


2 


38 


% of correct 


97.0 


88.7 


86.4 


0.0 


90.0 


The Old Man with 


a Won (104 sentences, 267 nouns) 


correct 


80 


137 


6 





223 


incorrect 


7 


32 


4 


1 


44 


% of correct 


92.0 


81.1 


60.0 


0.0 


83.5 


an essay tensei jingo (23 sentences, 98 nouns) 


correct 


26 


40 


17 





83 


incorrect 


4 


9 


2 





15 


% of correct 


86.7 


81.6 


89.5 




84.7 


average 












% of appearance 


29.1 


57.7 


12.8 


0.4 


100.0 


% of correct 


93.6 


84.9 


84.2 


0.0 


87.0 



Table 6. Machine Learning 2 (test set) 





indcf 


def 


gen 


other 


total 


a folktale turu (263 sentences, 699 nouns) 


correct 


112 


360 


13 





485 


incorrect 


35 


163 


6 


10 


214 


% of correct 


76.2 


68.8 


68.4 


0.0 


69.4 


an essay tensei jingo (75 sentences, 283 nouns) 


correct 


79 


88 


14 





181 


incorrect 


37 


53 


12 





102 


% of correct 


68.1 


62.4 


53.9 




64.0 


Pacific Asia (22 sentences, 192 nouns) 


correct 


25 


110 


10 





145 


incorrect 


13 


29 


3 


2 


47 


% of correct 


65.8 


79.1 


76.9 


0.0 


75.5 


average 












% of appearance 


25.6 


68.4 


4.9 


1.0 


100.0 


% of correct 


71.8 


69.5 


63.8 


0.0 


69.1 



learning set is multiplied by the inverse of its occurrence. For example, in this 
paper, we multiplied 4, 2, and 9 by the frequencies of "indefinite," "definite," 
and "generic." 

In other words, since generic noun phrases only made up 2/9 of definite 
noun phrases, we changed the frequencies of data as if definite noun phrases 
had occurred twice as often as in the actual data and generic noun phrases had 
occurred nine times more often than in the actual data. We found that this 
change made the frequencies of the three referential properties uniform and did 
not bias the analysis towards definite noun phrases. The change produced the 
following result. Machine-Learning 1 applies a general method of learning and 
learns data in order to maximize the following equation. 

evaluation function = [% of correct in overall the data] (6) 

On the other hand, Machine-Learning 2 uses the frequencies of the referential 
properties and learns data in order to maximize the following equation. 

evaluation function 
=the average of 
[% of correct in "indefinite"], (7) 
[% of correct in "definite"] and 
[% of correct in "generic"] 

The results for Machine-Learning 2 are listed in Tables ^ and ^. 

The accuracy rate of Machine-Learning 2 on the test set was 69.1%. This 
is nearly equal to the 68.9% obtained by using the Manual Adjustment. It was 
found that the accuracy rates for all three referential properties were about 
70%. Since even for the worst category, "generic," 63.8% was achieved, it is clear 
that Machine-Learning 2 was able to quite precisely estimate the referential 
properties. 

As the above results show, we found that manual adjustment for estimat- 
ing referential properties was not necessary and therefore human costs were 
decreased. 

We also examined the values of the rules as given by the maximum entropy 
method used in Machine-Learning 2. We examine some of the rules as listed 
below. In each of the rules in the list, (i) the condition parts, (ii) values assigned 
by hand, and (iii) values assigned by Machine learning 2 are included. 

1. Rules for indefinite noun phrases 

(a) When a noun is accompanied by a particle, ga (new-topic marker), then 
{indefinite (1, 2) definite (1, 1) generic (1, 0)} 
{indefinite 0.62, definite 0.21, generic 0.17} 

In general, a noun accompanied by a particle, ga, one function of which is 
to indicate a new topic, roughly tends to be indefinite. The highest value 
is thus assigned to "indefinite" when manual adjustment is used. The value 
of "indefinite" as determined by Machine-Learning 2 is also the highest. In 



manual adjustment, the possibilities of all categories are set to 1, so any of 
the three categories can be the answer. In Machine-Learning 2, since the 
value of "indefinite" is not terribly high, (not, e.g., 0.99), any of the three 
categories can also be the answer. 

(b) When a noun is modified by an adjective aru (a certain ~), then 

{indefinite (1, 2) definite (0, 0) generic (0, 0)} 
{indefinite 0.99, definite 0.0001, generic 0.0001} 

Generally, a noun modified by aru (a certain ~) is indefinite. The values for 
possibility of the other categories are thus set to in manual adjustment. 
The values used in Machine-Learning 2 are 0.0001, which is also a very small 
value. The value of "indefinite" is extremely high in Machine-Learning 2. We 
find that Machine-Learning 2 can judge that the category of a noun modified 
by aru is almost certainly indefinite. 

2. Rules for definite noun phrases 

(a) When a noun is a pronoun, then 
{indefinite (0, 0) definite (1, 2) generic (0, 0)} 
{indefinite 0.005, definite 0.99, generic 0.005} 

When a noun is a pronoun, it is always definite, so the values of possibility of 
the other categories arc manually set to 0. The values assigned to the other 
categories by Machine-Learning 2 arc also very small. 

(b) When a noun is modified by an embedded sentence which has a definite 
noun accompanied by wa or ga (nominative-case particle), then 
{indefinite (1, 0) definite (1, 1) generic (1, 0)} 

{indefinite 0.19, definite 0.61, generic 0.19} 

Although such a noun is not always definite, it is likely to be definite. The 
value of "definite" in Machine-Learning 2 is the highest of the three, but is 
not extremely high. 

3. Rules for generic noun phrases 

(a) When a noun is followed by a particle wa, which does not have a modifier, 
then 

{indefinite (1, 0) definite (1, 1) generic (1, 1)} 
{indefinite 0.03, definite 0.26, generic 0.71} 

The particle wa, which is a topic marker, is an expression which is likely 
to accompany either a definite noun phrase or a generic noun phrase. The 
possibility values of the two categories are both set to 1 by Manual Adjust- 
ment. Machine-Learning 2 found that the values assigned to "definite" and 
"generic" are higher than that assigned to "indefinite," but "generic" is as- 
signed a higher value than "definite." This is because there are many rules 
for estimating that a noun phrase is "definite," so a word can be estimated 
as "generic" if no other clue words appear. 

(b) When a noun is followed by a particle wa and it modifies an adjective, 

then 

{indefinite (1, 0) definite (1, 3) generic (1, 4)} 
{indefinite 0.13, definite 0.80, generic 0.07} 



Although "generic" is assigned the highest value by Manual Adjustment, 
"definite" is assigned the highest value by Machine-Learning 2. This is be- 
cause of (i) a wrong estimate by Machine-Learning 2 due to a small learning 
set, (ii) the influence of other rules, such as the previous rule, or (iii) incor- 
rect manual adjustment in the earlier work. If the actual reason is (i), then 
making the learning set larger should improve the results. 

As stated above, the values assigned by Machine-Learning 2 tended to be sim- 
ilar to those obtained by hand, and they demonstrated some degree of linguistic 
intuition. 

4 Conclusions 

We have succeeded in creating a system for automatically giving rules values for 
solving conflicts when estimating the referential properties of noun phrases. We 
have thus shown that the cost in terms of human time of manually adjusting 
values is not unavoidable. We also found that, in machine learning, making 
the frequencies of the categories uniform can help to make their accuracy rates 
more uniform. Finally, we examined the values produced for rules by applying 
the Machine-Learning method, and conflrmed that they were consistent with 
linguistic intuition. 
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